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1.  Introduction 

Since  the  second  world  war,  spurred  by  the  development  and  availabil¬ 
ity  of  computers  there  has  been  an  Increased  effort  directed  toward  developing 
methodology  for  solving  optimization  problems.  Problems  for  solution  have 
originated  from  two  groups.  The  first,  composed  of  economists,  operations 
research  analysts,  management  scientists,  and  planners  has  been  concerned  with 
the  problem  of  how  best  to  allocate  scarce  resources.  Since  the  functional 
relationships  in  this  area  are  not  known,  this  group  has,  in  the  main,  been 
content  to  develop  programming  models  having  only  linear  relationships.  The 
success  of  computer  codes  implementing  versions  of  the  Simplex  method  for 
solving  linear  programming  problems  has  reinforced  the  basic  orientation  of 
this  group. 

The  second  group,  with  a  longer  history.  Is  composed  of  physicists, 
chemists,  engineers,  and  other  scientists.  The  problems  of  interest  to  this 
group  contain  functional  relationships  which,  because  of  physical  laws,  are 
exact,  and  which  contain  nonlinear  effects.  Engineering  optimization  problems 
are  usually  ones  of  design.  Although  much  thought  and  time  has  been  put  into 
the  geometric  and  algebraic  descriptions  of  these  problems,  little  has  been 
done  in  a  systematic  way  to  attack  these  problems  using  recent  algorithmic 
and  computational  advances  in  nonlinear  programming. 

It  is  the  intent  of  this  paper  to  show  how  this  might  be  done. 
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Sensitivity  Analysis 


Rarely  is  a  person  with  a  design  model  interested  in  the  solution  to 
one  optimization  problem.  Of  equal  interest  usually  are  changes  in  the  op¬ 
timal  design  which  obtain  when  various  aspects  of  the  model  are  altered. 

One  way  to  obtain  this  information  is  to  solve  many  optimization  pi.  o’  i  "me 
containing  systematic  variations  of  the  input  parameters  of  the  model  and 
observe  the  changes  in  design. 

Another  way,  based  on  recent  developments  in  the  theory  characterizing 
a  solution  to  an  optimization  problem  is  to  use  information  available  at  the 
solution  to  a  particular  problem  and  generate  a  matrix  which  can  be  used 
Independently  to  assess  all  the  design  changes  affected  by  small  simultaneous 
input  parameter  changes.  The  theory  of  this  approach  is  contained  in  [3,  Sec¬ 
tion  2.4].  The  practial  use  of  this  approach  is  demonstrated  by  the  following 
example  taken  from  Paacual  [7,  p.  24],  and  Johnson  [4,  pp.  290-291]. 

Example  1  [Optimal  Design  of  a  Cylindrical  Torsion  Bar] 

The  problem  is  to  find  the  diameter  (d)  and  the  length  (L)  of  a  cylin¬ 
drical  torsion  bar  which  will  ‘transmit  a  virtually  constant  twisting  torque 

(T),  have  a  specified  torsional  rigidity  (k),  and  for  which  the  weight  of  the 
bar  will  be  minimal  over  all  those  having  these  properties. 

The  mathematical  programming  problem  is 

minimize  W  ■  wjrd^L/4  (1) 

(d,L) 

subject  to  the  constraints 

16T/ird3  <  St/2N  ,  (2) 


k  -  ird4C/32L 


(3) 
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Here 

W  ■  weight  of  the  torsion  bar, 
d  ■  diameter  of  the  torsion  bar, 

L  ••  length  of  the  torsion  bar, 
w  «  specific  weight  of  the  torsion  bar, 

G  -  modulus  of  elasticity  in  shear  (or  modulus  of  rigidity) , 

St  -  published  yield  strength  of  the  material, 

N  •  given  factor  of  safety  based  on  the  occurrence  of  yielding  as  the 
failure  phenomenon, 

T  ■  the  torque,  and 

k  -  the  prescribed  torsional  rigidity. 

The  expression  for  weight  in  (1)  is  simply  the  volume  of  the  cylin¬ 
drical  torsion  bar  times  the  specific  density.  The  expression  on  the  left  of 
(2)  gives  the  maximum  state  of  shear  stress  experienced  by  the  bar  for  the 
applied  torque  I  .  This  occurs  at  the  outer  edge  of  the  bar  and  is  the  same 
along  its  length.  This  must  not  exceed  the  experimentally  determined  yield 
point  of  the  bar  or  permanent  distortion  will  result.  Equation  (3)  gives  the 
torsional  rigidity  which  is  a  function  of  the  diameter  of  the  bar  as  well  as 
its  modulus  of  rigidity.  The  higher  the  k  value  the  less  temporary  defor¬ 
mation  is  experienced  by  the  bar  when  torque  Is  applied  to  it. 

3 

For  this  model,  values  of  the  parameters  used  were  w  •  490  lbs /ft  , 

G  -  11.5  x  106  psi,  Sc  -  35,000  psi,  N  -  2,  T  -  1000  Ib-irt,  snd  k  -  105  lb-in. 

For  purposes  of  convenience  the  problem  is  rewritten 

2 

minimize  wird  L/4 
(d,L) 

subject  to  xd^S  -  32NT  >  0  ,  (4) 

U 

32Lk  -  xdAG  -  0  , 

-  3  - 


(5) 
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.  *  * 

The  solution  to  this  problem  occurs  at  (d  ,L  )  -  (.834937,5.486689,, 
(both  quantities  in  inches).  The  generalized  Lagrange  multiplier  associated 
with  the  inequality  constraint  is  u^  ■  .186342  x  10  " ,  and  the  Lagrange 

A 

multiplier  associated  with  the  equality  constraint  has  a  value  w^  •  .485175 
x  10  6  .  These  are  the  values  for  which  the  gradient  of  the  Lagrangian  func¬ 
tion  2  3  U 

ot(d,L,u1,w1)  -  wnd  L/4  -  u1(ird  St~32NT)  +  wJL(32Lk-Trd4G) 

vanishes,  i.e,,  for  which 


wmdL/2 

37rd2St* 

u,  + 

r  3  *i 

-4rd  G 

w,  * 

“  «■ 

0 

wird^/4 

0 

1 

32k 

1 

0 

■  m 

The  interpretation  of  the  Lagrange  rjitiplier  is  well-known.  If  the 

right  hana  side  of  (4)  is  changed  to  6  ,  then  the  solution  to  the  perturbed 

problem  has  an  optimal  objective  function  value  which  differs  from  that  of 

* 

the  problem  just  solved  by  approximately  6u^  .  What  is  not  so  well-known  is 
that  information  is  available  to  answer  the  question  of  how  the  minimizing 
point  Itself  changes  when  general  perturbations  are  placed  on  the  above 
problem. 

To  illustrate  this  without  giving  the  general  formulation  which  is 
contained  in  [3,  section  2.4],  suppose  the  question  was  posed  for  the  above 
example  how  much  would  the  solution  change  if  the  torsional  rigidity  require¬ 
ment  (k)  were  changed  from  100,000  lb-in  to  102,000  lb-in  and  simultaneously 
the  torque  (T)  from  1000  lb-in  to  1030.301  lb- in  ? 

From  the  general  theory  in  [3]  we  obtain  the  result  that  the  problem 
resulting  from  the  replacement  in  (4)  of  T  by  T  +  AT  ,  and  in  (5)  k  by 
k  +  4k  results  in  a  new  minimizing  vector  and  associated  Lagrange  multipliers 
which  can  be  approximated  from  those  of  the  original  problem  by  the  expression 


-  4  - 
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*1 

a*  A  2  *  * ,  2  .*<3 

~ 

d 

wit L/ 2— 6tt d  Stu1-12(d  )  Gw^  wird  /2,  -3n(d  )  S  ,  -4n(d  )JC 

0,  0 

* 

A , 

A 

L 

wrd  /2  ,  0  ,  0  ,  32k 

0,  32w. 

X 

* 

+ 

A  A  2  „ 

A 

U1 

u1<.3TT(d  )  St)  ,  0  ,  0  ,0 

64up  0 

* 

*  3 

A 

WL 

-4tf(d  )  G  ,  32k  ,0,0 

0,  -32L 

•• 

. 

Using  AT  -  30.301,  and  Ak  ■  2,000,  computation  of  the  above  quantities 
yields  the  expected  change  in  the  optimal  d  and  L  as  (.00843,, 11193)  . 

For  comparison  purposes,  the  perturbed  problem  has  a  solution  which  differs 
from  the  original  one  by  (.00834, .11083) . .  Thus  the  approximation  agrees 
with  the  exact  change  to  about  3  significant  figures. 

In  some  problems  it  is  not  £  priori  obvious  even  the  direction  of 
change  of  the  problem  solution  when  the  values  of  many  parameters  are  changed 
simultaneously.  The  use  of  sensitivity  information  should  have  wide  appli¬ 
cation  in  the  area  of  engineering  design. 
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3.  Obtaining  Global  as  Opposed  to  Local  Solutions  by  Use  of  Branch  and 
Bound  Techniques 

Most  algorithms  for  solving  nonlinear  programming  problems  are  only 
guaranteed  to  find  global  (as  opposed  to  local)  solutions  when  the  problem 
functions  describe  a  convex  programming  problem.  Problems  which  can  trap  an 
algorithm  at  a  local  solution  often  arise  when  the  objective  function  (to  be 
minimized)  is  a  total  cost  function  with  decreasing  marginal  costs.  A  simple 
picture  is  contained  in  Figure  1.  The  curve  shown  in  Figure  1  is  concave 
whereas  if  it  were  convex,  its  use  in  an  optimization  problem  would  cause  no 
difficulty.  To  illustrate  the  difficulties  this  can  cause  and  also  to  illus¬ 
trate  the  increasingly  important  branch  and  bound  approach  for  solving  these 
problems  we  consider  the  following  example.  It  is  taken  from  the  area  of 
water  pollution  control  which  is  becoming  a  source  of  optimization  problems 
which  Involve  both  groups  of  people,  the  social  and  physical  scientists  men¬ 
tioned  in  the  introduction. 

Example  2  [Optimal  Reservoir  Design] 

The  concentration  of  dissolved  oxygen  in  the  water  of  a  certain  section 
of  a  river  is  below  the  acceptable  level  due  to  pollution  of  the  river.  It 
is  desired  to  increase  the  level  of  dissolved  oxygen  to  the  acceptable  level 
by  building  reservoirs  at  one  or  more  upstream  points  of  the  river  so  as  to  be 
able  to  augment  the  river  flow  and  dilute  the  wastes. 

Although  the  original  model  contained  many  sites,  for  illustrative 
pusposes  we  consider  an  idealized  model  with  two  sites.  Let  x,  be  the  flow 

J 

25 

augmentation  at  site  j ,  j  ■  1,2,  and  lx^  be  the  cost  to  build  a  reservoir 

25 

at  site  1  to  augment  the  flow  by  amount  ,  and  let  2xj  be  the  cost  to 
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build  a rasarvolr  at  sice  2  to  augment  the  flow  by  amount  •  Assume  that 
the  required  augmentation  is  3.  Algebraically  stated  then  the  optimization 

A  A 

problem  is  to  find  and  which  minimize  the  total  cost 

lx*5  +  2x*25 

subject  tv',  the  constraints 

X1  +  x2  >  3,  x^  >  u,  x2  >  0,  x^  <  6,  x2  <  5  . 


l’.ie  region  of  feasible  points  along  with  two  curves  of  constant  costs  are 
plotted  in  Figure  2. 

It  is  obvious  that  (0,3)  and  (3,0)  are  local  solutions  to  the  problem, 
l.e.,  that  in  a  region  about  either  point  all  information  indicates  that  that 
point  is  a  solution-  Just  about  any  algorithm  which  tries  to  solve  this  prob¬ 
lem  will  go  to  either  solution  depending  upon  the  initial  starting  point.  A 
uarantee  of  the  ability  of  an  algorithm  to  obtain  a  global  (as  opposed  to 
local)  solution  is  required. 

Recently  branch  and  bc-md  methodology  [1],  [9]  has  been  applied  to 
problems  of  the  above  form  in  order  to  guarantee  convergence  to  a  global  solu¬ 
tion.  Rather  than  define  what  a  branch  and  bound  method  is  in  general,  the 
problem  just  stated  will  be  solved  by  such  an  algorithm. 

The  basic  branch  and  bound  approach  for  solving  the  nonconvex  program¬ 
ming  problem  is  to  solve  a  sequence  of  underestimating  convex  programming 
problems  which  apply  to  subregions  of  the  feasible  region.  A  sequence  of 
underestimating  values  for  the  solution  of  the  original  problem  is  created 
which  approaches  the  optimal  value  from  below.  First  a  convex  (in  this  case 
linear)  underestimate  is  generated  of  the  objective  function  in  the  rectangle 
0  -  -  b,  0  <  x^  <  5  .  Figures  3  and  4  show  that  the  linear  function 

.408x^  is  a  linear  underestimate  of  the  function  x^5  in  the  Interval 
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Concave  Function  of  x.  with  Underestimating 
Convex  Envelope 
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[0,6]  ,  and  that  .598x2  is  a  linear  underestimate  of  2x^  in  the  interval 
[0,5]  .  In  fact  these  straight  lines  are  the  highest  convex  functions  which 
underestimate  the  original  ones  in  those  intervals. 

The  following  programming  problem  is  an  underestimating  convex  (linear 
in  this  case)  programming  problem  for  the  original  nonconvex  one.  Find 
values  of  x^  and  which 

minimize  ,408x^  +  .598x2 

subject  to  the  constraints 

xL  +  x2  >  3,  0  <  xx  <  6,  0  <  x2  <  5  . 

* 

The  solution  to  this  problem  is  at  x  *  (3,0)  ,  with  an  objective  function 
value  (of  the  linear  underestimating  function)  of  1,224,  Thus  we  know  that 
the  global  minimizing  value  to  the  original  problem  is  bounded  below  by  1.224. 

The  next  step  in  the  procedure  is  to  divide  the  original  rectangle  into 
two  parts  and  solve  an  underestimating  problem  for  each  part  which  is  'closer 
to'  the  original  problem  than  the  first  underestimating  problem  solved  above. 
The  decision  on  how  to  divide  is  made  by  choosing  that  variable  for  which  the 
difference  in  function  values  between  the  underestimating  function  and  the 
original  function  is  greatest.  Using  this  criterion,  the  decision  is  made  to 
split  the  original  rectangle  into  two  parts  on  x^  ,  yielding  the  new  sets  of 
constraints  0  <  x^  <  3,  0  <  x2  <  5,  and  3  <  x^  <  6,  0  <  x2  <  5  .  Using  the 
same  approach  for  generating  convex  underestimating  envelopes  the  first  new 
programming  problem  to  be  solved  is 

minimize  ,577x^  +  . 593x2 
subject  to  0  <  <  3,  0  <  x2  <  5,  +  x2  >  3 

which  has  a  solution  at  (3,0)  with  solution  value  1.732. 
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The  problem  associated  with  the  other  rectangle  Is 

minimize  1.015  +  . 239x^  +  .598x2 
subject  to  3  <  <  6,  0  <  X2  <  5,  x^  +  X2  >  3 

which  has  a  solution  at  (3,0)  with  solution  value  1.732  also.  Since  both 
values  are  lower  bounds  to  the  global  optimum,  and  since  (3)’^  +  2(0) « 
1.732  ,  the  global  solution  has  been  obtained. 

Branch  and  bound  techniques  can  also  be  used  to  handle  problems  which 
are  not  well-behaved  in  that  the  functions  lack  the  proper  smoothness  or 
continuity  properties.  Also,  these  techniques  to  date  are  the  most  successful 
approaches  for  solving  combinatorial  problems,  e.g.,  problems  where  the  diffi¬ 
culty  is  in  ascertaining  the  order  in  which  activities  are  to  be  performed, 
or  whether  or  not  to  perform  them  rather  than  the  optimal  level  of  any  activ~ 
ity. 
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4.  Computer  Code-Optimization  Model  Interface 

A  major  hurdle  to  the  computer  solution  of  nonlinear  optimization 
problems  in  engineering  design  is  the  lack  of  an  automated  procedure  for 
giving  codes  all  the  Inputs  an  algorithm  needs  to  solve  the  problem.  For 
example, the  branch  and  bound  procedure  applied  to  the  problem  in  the  previous 
section  is  general  enough  to  solve  separable  programming  problems  of  the  form 

minimize  fj(x^) 

subject  to  gi^(Xj)  >  0,  i  -  l»...,m  . 

A  further  requirement  is  that  for  any  function,  say  fj(x^)  *n  any  given 
interval  [L ^  ,U^  ]  ,  the  user  must  supply  a  function  dj(x^)  which  is  the 
convex  envelope  of  fj(x^)  *n  t*ie  interval.  That  is,  one  must  supply  to  the 
computer  code  implementing  the  branch  and  bound  method  the  highest  convex 
envelope  which  underestimates  the  function  f^(Xj)  chat  interval. 

For  another  example,  the  code  implementing  the  sequential  unconstrained 
minimization  technique  (S1JMT)  15],  [3]  requires  the  uBer  to  supply  the  first 
derivatives  of  the  problem  functions.  To  be  efficient,  the  code  asks  the  user 
to  supply  the  second  derivatives.  A  certain  amount  of  this  trouble  can  be 
avoided  by  the  use  of  difference  approximations  instead  of  the  analytic  forms, 
but  it  is  no  wonder  that  engineers  in  the  design  area  are  more  inclined  to  use 
heuristically  based  algorithms  such  as  the  SIMPLEX  SEARCH  method  [6]  for 
solving  their  optimization  problems  which  require  only  function  values. 

One  solution  to  this  problem  has  been  the  development  of  a  language  [8] 
and  a  computer  program  which  automatically  computes  first  and  second  deriva¬ 
tives  of  an  algebraic  expression  when  the  problem  functions  are  written  down 
in  the  language  by  the  user.  In  (6)  is  an  expression  giving  the  first  stage 
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velocity  of  a  three  stage  space  launch  vehicle  as  a  function  of  design  vari¬ 
ables  written  in  this  language.  The  FORTRAN- like  expression  is 

X(14)*T1*GBAR*L0G(1./X(13))/X(15)  .  (6) 

where 

X(14)  "  stage  1  total  thruat  in  thousands  of  pounds , 

T1  -  stage  1  bum  time  in  seconds , 

X(13)  ■  stage  1  masa  fraction  (dimensionless) , 

GBAR  -  gravity  constant,  and 
L0G  -  natural  logarithm  function. 

The  first  four  quantities  above  are  design  variables  of  the  optimization  model 
[1,  Chapter  2],  which  was  to  produce  a  minimum  coat  design  satisfying  certain 
performance  characteristics.  Its  solution  using  SUMT  required  the  supplying 
of  a  computer  program  for  the  analytic  derivatives  of  expressions  like  the 
one  in  (6),  This  was  an  extremely  time  consuming  and  an  error  prone  enter¬ 
prise.  Using  the  new  language,  such  model  Implementation  is  much  more  feasible. 

Another  approach  to  this  problem  now  under  consideration  ia  to  first 
convert  the  programming  problem  into  a  separable  problem  and  devise  a  card 
format  for  separable  terms  allowing  for  any  convelvable  functions  of  a  single 
variable.  Computer  routines  could  be  coded  and  used  to  compute  the  deriva¬ 
tives,  first  and  second,  and  the  convex  envelopes  of  functions  of  a  single 
variable. 

The  general  procedure  for  converting  an  optimization  problem  to  a 
separable  problem  Is  to  add  variables  and  increase  the  number  of  constraints. 

As  an  example,  consider  the  following  which  expresses  the  requirement  that 
the  section  modulus  of  a  vertical  transverse  corrugated  bulkhead  satisfy 
certain  rulas  contained  in  Pet  Norake  Veritas.  Th«  inequality  is  [1,  p.  56], 
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1  .  *  ,  «2  f  .  /  2  2..1/2,  »  n 

J  x2»3x^  +  J  X1X3X4  “  kht*t  [x^  +  (x2-x3)  J  i  o  . 

This  constraint  can  be  ’separated'  by  the  addition  of  three  more  variables 

and  three  constraints.  The  resulting  constraints  are 

2 

lnx^  +  lnx^  +  Xj  -  ln(kht!t)  -  lnx^  t  0  , 

X1  +  *6  "  x4 
1  .  a  x. 

6  x2  +  2  X1  "  *  5 

2  2  2 
x2  -  x3  -  x6  . 

‘  A  computer  code  can  probably  be  developed  to  take  the  functions  written 
In  the  language  described  In  [8]  and  do  the  conversions  required  to  separate 
the  problem. 
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5.  Conclusion 


In  this  paper,  a  brief  summary  of  some  recent  powerful  theoretical 
toola  for  sensitivity  analysis  in  engineering  design,  and  recent  globally 
convergent  algorithms  for  solving  optimization  problems  have  been  given. 

The  barrier  to  using  these  tools  and  other  recent  algorithms  for  solving 
problems  is  the  lack  of  a  readily  available,  easily  used  language  in  which 
to  describe  the  optimization  models.  T> '  1 anguage  developed  by  Pugh  [8]  is 
a  step  toward  eliminating  this  barrier.  '  .ture  progress  will  depend  in 
large  part  on  bringing  together  those  with  the  algorithmic  ability,  those 
with  the  design  optimization  problems,  and  those  with  the  capability  of 
developing  computer  systems. 
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